Conceptualization and Measurement

Social science is about developing and evaluating causal theories

• Does economic development lead to democratization?

• Do gender quotas cause an increase in female representation?

Theories are comprised of linkages between concepts

• Gender Quotas, electoral institutions, women’s CSOs, public opinion, and political corruption

• Causal relationships are comprised of concepts that influence one another

• Important to consider how we measure these concepts

Concepts are theoretical abstractions

• Sometimes measurement is straightforward

  • What is your age?

  • What state do you live in?

• Other times measurement is more complicated

  • What is political corruption?

  • What is female representation?

  • What is democracy?

• Have to develop an operational definition of a concept before we can measure it

Descriptive research is comprised of three primary goals

• To answer research questions

• To generate research questions

• To do both of these iteratively

Yet, it is necessarily inferior to causal approaches because it cannot answer questions of why or how things happen

Descriptive Question: Who Votes for Trump?

Causal Question: Why do people vote for Trump?

Descriptive Question: What is media bias?

Causal Question: Does media bias influence political attitudes?

A rectangular, observation-by-variable dataset

• “dataset observations” (DSOs)

A clear unit of analysis

• Country-years, state-months, respondent-year, etc.

Requires multiple cases/units

• Nothing can be learned from an \(\sample{n}\) of 1

Quantitative and qualitative measures

• In the social sciences quantitative variables are measures of qualitative concepts

Calculation of summary statistics

Measure features

• Need to consider level of measurement and how to score each case on each feature

• To study concepts, we need to be able to observe those concepts and encode them as variables

• Can’t study things we can’t measure and can’t measure things we can’t observe

  • Exception to this is latent variables that we can’t observe but can try to measure

The definition of a variable

• A dimension that describes an observation or the operationalization of a concept

How should we measure democracy?

• Requires we first define democracy

• One of the oldest debates in comparative politics

  • Current gold standard is V-Dem (Coppedge et al. 2016; Lindberg et al. 2014), but even it is imperfect and leaves room for improvement

• Sometimes the worst, except for all the others is the best we can do

• As a general rule, treating complex multi-dimensional concepts as dichotomies is at best logically fallacious

• Need to make principled tradeoffs between parsimony and precision

Conceptual Clarity

• Conceptual Clarity is about knowing what we want to measure

• Poorly defined concepts lead to poorly measured variables

  • Ambiguity

  • Vagueness

Revise concept definition as needed

• But revisions have to made before your analysis

Construct Validity

• Construct Validity is the degree to which a variable measures a concept

• Does our variable actually measure what we say it measures or some secret third thing?

• Construct validity is high if a variable adequately measures the concept we care about

• Construct validity is low if a variable is actually a measure of something else

Common measure of democracy that was dominant in IR and Comparative Politics for several decades

Mostly a measure of institutional constraints on the executive

Are institutional constraints on the executive and democracy the same thing?

• There are people who think stupid things, some of them in this department, who would argue yes

• As we established earlier, these people are wrong

Polity is the QWERTY keyboard of political science

• An inferior product that became entrenched by being first to market

The measures we use often influence the answers we get

As a general principle, the result of any operation performed on a precise number and garbage is some variation of garbage

• \(\mathrm{Precise~Number} \times \mathrm{Garbage} = \mathrm{Garbage}\)

• \(\sqrt{\mathrm{Garbage}} = \mathrm{Less~Bad~Garbage}\)

• \(\mathrm{Garbage}^{2} = \mathrm{Worse~Garbage}\)

• \(\frac{1}{\sample{n}}\sum_{\obs{i}=\obs{1}}^{\sample{n}}[ \mathrm{Garbage}_{\obs{i}}] = \mathrm{Better~Garbage}\)

• \(\mathrm{Precise~Number}^{\mathrm{Garbage}} = \mathrm{Much~Worse~Garbage}\)

• \(\mathrm{E[\resp{Y} _{\obs{i}} = Garbage~|~\treat{X}_{\obs{i}}=Precise~Number}] = \mathrm{Expectation~of~Garbage}\)

• \(\resp{y} _{\obs{i}} = \alpha + \hat{\beta_{1}}\times\mathrm{Polity~IV}_{\obs{i}\tim{t}} = \mathrm{Comparative~Garbage}\)

Most thick concepts are multidimensional

Things look for

• Convergence (Convergent Validity)

• Discrimination (Discriminant Validity)

• For example, the multi-trait, multi-method matrix

Once we have measured variables for observations, we can conduct analysis!

• And not a damn moment sooner

And once we have analysis, we can draw inferences and make evidence-based claims

• Political science is a field interested in evaluating causal theories.

• No one care what you think unless you back it up with empircally valid evidence

We want to make claims about concepts

But we only observe and can only analyse observed, measured variables

So our task as social scientists is to

• Link the concepts we care about to observable phenomena

  • Components, indicators, variables

  • Draw out theoretical implications from measures

Good measurement is theory-driven